An efficient approach for mining web content sensitivity

Abnormal remarks on the web, such as violence, threat, superstition, etc., may disturb the social order and public morality (referred as sensitive content). To provide a quantitative measure of the sensitivity of a webpage, we propose the concept of web content sensitivity which measures how sensitive a page is. We also propose a web content sensitivity mining approach. Our experiment identified a number of sensitive webpages that traditional frequency-based methods failed to find. By varying the sensitive values of the keywords, different sets of high sensitivity keywords were discovered as well as the corresponding webpages.